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SUMMARY  PAGE 


THE  PROBLEM 

inis  investigation  concerns  the  validity  of  the  officer-like  qualities  (OLQ)  rating. 

It  deals  with  measures  predictive  of  adequacy  as  an  officer  subsequent  to  training 
(i.  e.,  in  tne  fleet).  The  report  considers  in  addition  to  the  formal  OLQ  such  meas¬ 
ures  as  delinquency  reports  and  demerits,  and  peer  nominations  of  leadersnip  and  social 
living.  The  properties  of  the  rating  system  are  also  analysed. 


FINDINGS 

The  OLQ  rating  significantly  differentiates  between  officers  judged  in  the  fleet  as 
satisfactory  and  as  unsatisfactory.  This  result  holds  for  the  over-all  score  obtained  in 
basic  and  advanced  training,  but  was  noted  in  pre-flight  only  for  certain  items.  Other 
measures  of  nontechnical  qualities  also  contribute  to  the  pattern  of  difference  tnrough- 
out  training  of  the  fleet  criterion  groups. 


RECOMMENDATIONS 

It  is  suggested  that  a  serious  effort  be  made  to  improve  the  assessment  of  officer-like 
qualities  both  in  the  direction  of  increasing  the  validity  of  measures  and  of  extending 
the  information  to  be  derived  from  these  measures.  Detailed  suggestions  are  given  in 
the  final  section  of  this  report. 


IT  HAS  BEEN  WELL  SAID  THAT  'HE  WHO  HAS 
NEVER  LEARNED  TO  OBEY  CANNOT  BE  A 
GOOD  COMMANDER'.  THE  TWO  ARE  NOT 
THE  SAME,  BUT  THE  GOOD  CITIZEN  OUGHT 
TO  BE  CAPABLE  OF  BOTH;  HE  SHOULD  KNOW 
HOW  TO  GOVERN  LIKE  A  FREEMAN,  AND 
HOW  TO  OBEY  LIKE  A  FREEMAN. . . . 


Aristotle,  in  The  Politics. 


INTRODUCTION 

The  process  of  training  a  young  man  to  become  a  naval  aviator  involves  the  twofold 
task  of  teaching  him  to  fly  and  preparing  him  to  serve  as  a  competent  naval  officer. 

By  this  time,  a  great  deal  is  known  about  how  to  achieve  the  first  of  these  objectives. 

In  the  second  respect,  however,  a  great  deal  is  left  to  the  unguidea  experience  of  the 
individual  as  he  lives  within  the  military  world  of  naval  aviation.  The  chief  purpose 
of  the  present  study  is  to  determine  the  validity  of  measures  of  officerlikeness.  We  do 
not  here  propose  to  examine  the  qualities  which  are  desirable  in  a  good  officer,  al¬ 
though  this  matter  is  one  which  could  and  perhaps  should  be  subjected  to  intensive 
investigation.  There  are,  of  course,  a  large  number  of  commonly  mentioned  character¬ 
istics  which  it  is  said  that  a  naval  officer  should  have,  but  these  have  not  been  clearly 
substantiated  in  objective  research.  It  is  at  least  reasonable,  however,  to  suppose  that 
whatever  the  specific  qualities  may  prove  to  be,  they  fall  into  three  or  four  fairly 
clearly  defined  categories,  namely,  1)  military  behavior,’  2)  leadership  and  command, 
and  3)  social  acceptability  within  the  military  system.  The  third  of  these  includes  both 
approval  by  peers  both  in  the  sense  of  acceptability  as  a  person  and  in  the  sense  of 
recognized  competence  and  devotion  to  duty;  further  study  might  lead  to  a  sufficiently 
sharp  distinction  between  these  to  warrant  their  separate  formulation,  it  is  not  possible 
to  treat  these  types  of  variables  separately  in  this  report,  but  the  general  tenor  of  the 
findings  suggests  that  future  consideration  of  the  problem  might  well  accept  them  as 
different  aspects  of  being  a  naval  officer,.  All  that  we  can  do  here  is  to  see  whether 
"officer-like  qualities"  can  be  measured  and  to  present  evidence  concerning  the  suc¬ 
cess  with  which  such  measurement  is  now  carried  out.  Although  we  are  not  directly 
concerned  with  questions  pertaining  to  the  training  of  a  person  as  an  officer,  it  may  be 
regarded  as  a  fundamental  assumption  that  this  aspect  of  training  has  great  importance 
in  addition  to  the  parallel  task  of  training  him  as  an  aviator.  The  former  may  be  called 


the  "non-technical"  or  "professional"  requirements  of  the  training  program,  compared 
to  the  "technical"  requirements  evident  in  ground  school  and  flight. 

Approximately  a  decade  ago  in  recognition  of  the  significance  of  nontechnical 
requirements,  an  effort  was  made  to  introduce  an  assessment  of  them  in  the  form  of  an 
"officer-like  qualities"  rating.  Over  the  years  this  form  has  regularly  been  included 
in  the  training  record  and  has  been  a  part  of  the  grading  system.  The  rating  is  obtained 
on  naval  aviation  cadets  at  each  of  the  three  stages,  pre-flight,  basic,  and  advanced. 

It  is  not  possible  to  determine  just  how  large  these  rating  loom  in  attrition,  but  it  is 
probable  that  relatively  few  men  are  actually  "washed  out"  because  of  unsatisfactory 
OLQ  ratings.  So  far  as  can  be  determined,  however,  no  detailed  study  of  the  OLQ 
rating  has  been  carried  out,  either  to  evaluate  it  as  an  assessment  technique  or  to 
ascertain  its  use  in  practice. 

There  are,  of  course,  a  number  of  research  studies  which  have  at  least  peripheral 
relevance  to  the  general  problem  of  assessing  persons  as  officers.  For  example,  it  has 
been  found  that  pre-flight  training  tends  to  make  cadets  less  "agreeable,"  more  in¬ 
clined  towards  "introverted  thinking,"  and  less  active  overtly  (10).  While  such  changes 
may  merely  reflect  the  character  of  the  stage  of  training  (for  instance,  they  may  be 
associated  with  the  fact  that  pre-flight  demands  intensive  academic  study),  they  in¬ 
dicate  the  possibility  that  important  personality  changes  occur  during  training,  in' 
addition  to  an  increase  in  technical  proficiency.  Investigation  has  also  shown  differ¬ 
ences  between  men  high  and  low  in  desire  to  remain  in  flight  training  (2,5).  Men  with 
stronger  desire  to  stay  in  the  program  display  a  stronger  interest  in  flying.  Further, 
those  with  stronger  desire  place  greater  value  on  inter-personal  relationships.  Such 
findings  are  suggestive  in  showing  that  success  in  aviation  training  places  a  premium 
on  certain  personal  characteristics  rather  than  on  others. 

Of  more  direct  pertinence  are  the  numerous  studies  of  peer  nominations  which  spe¬ 
cifically  concern  the  qualities  which  are  presumed  to  be  important  in  an  officer,  not¬ 
ably  leadership  (see,  for  example,  1,6).  It  would  serve  no  useful  purpose  to  enter 
upon  a  review  of  this  large  volume  of  research  which  has  been  carried  out  in  other 
branches  of  the  military  service  in  addition  to  the  Navy.  Suffice  it  to  say,  that  peer 
nominations,  in  which  the  men  in  a  class  are  asked  to  make  judgments  about  their 
fellow  cadets  in  some  specific  respect,  are  becoming  widely  accepted  as  an  easy  and 
useful  means  to  assess  non-technical  qualities.  The  logic  of  the  procedure  rests  pri¬ 
marily  upon  the  fact  that  a  group  of  cadets  have  a  better  opportunity  to  observe  and 

*  Later,  we  shall  present  evidence  on  the  validity  of  peer  nominations  as  predictions  of 
ultimate  satisfactoriness  as  an  officer. 
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evaluate  each  other  than  does  a  commanding  officer,  or  a  classroom  instructor*  An 
outgrowth  of  the  peer  nomin  'tior.  approach  has  been  to  examine  the  characteristics  of 
tnose  who  are  hignly  chosen  or  rejected  by  their  associates.  One  important  finding 
,ias  been  tnat  the  individual  regarded  as  an  effective  leader  tends  to  be  "antonomous" 
ratuer  than  "authoritarian"  in  character  (3,5,7).  In  effect,  this  means  that  the  cadet 
chosen  by  his  fellow  as  a  good  leader  is  likely  to  be  independent  and  self-assured, 
ratuer  than  conforming  and  uncritically  submissive  to  authority. 

Finally,  brief  mention  should  be  made  of  a  study  of  the  OLQ  itself  (9).  An  attempt 
was  made  to  compare  the  consistency  in  ratings  from  one  stage  of  training  to  the  next. 
Only  a  slight  agreement  was  found;  in  fact,  a  man  rating  near  the  top  at  one  stage, 
say  pre-flight,  might  actually  be  near  the  bottom  at  the  next  stage,  i.e.,  basic.  It 
is  not  clear  from  this  study,  however,  why  the  inconsistencies  occur.  It  is  possible, 
for  instance,  that  different  factors  are  measured  at  different  stages,  or  that  the  judg¬ 
ment  made  at  one  stage  is  undependable.  As  a  result  of  such  conditions,  a  sequential 
comparison  might  not  be  meaningful.  It  should  be  noted  that  even  if  the  inconsistency 
did  exist,  it  would  not  necessarily  mean  that  the  OLQ  at  each  stage  does  not  predict 
ultimate  success  as  a  naval  officer;  that  is,  an  independent  criterion  of  such  success 
might  show  a  significant  relation  to  the  OLQ  at  each  stage,  even  if  the  three  separate 
ratings  are  not  related  to  each  other.  Fortunately,  an  independent  criterion  was  avail¬ 
able  for  use  in  the  present  study  (8)  which  permits  a  reasonably  conclusive  test  of  this 
possibliiity . 

THE  CONTEXT  OF  THE  OLQ 


Before  we  proceed  with  a  detailing  of  results,  it  might  prove  useful  to  give  brief 
attention  to  the  properties  and  administration  of  the  OLQ  rating. 

An  example  is  given  in  Appendix  A.  In  general,  the  form  is  typical  of  devices  for 
rating  other  people  in  specified  characteristics.  It  has  eight  items  which  as  can  be 
seen,  can  be  divided  into  the  three  categories  previously  mentioned,  namely,  I)  mili¬ 
tary  behavior,  items  I  (military  courtesy),  2  (military  behavior),  and  3  (military  drill); 
2)  leadership  and  command,  item  8,  (leadership);  and  3)  social  acceptability,  items 
4  (initiative),  5  (social  adeptness),  6  (stabi!ity,)and  7  (character).  In  this  third  area 
the  items  are  not  as  clear  in  their  intentionas  in  the  other  two  since  quality  as  a  person 
and  devotion  to  duty  (or  identification  with  the  officer  role)  should  probably  be  differ¬ 
entiated.  Each  item  can  be  checked  as  "not  observed,"  "unsatisfactory,"  "below 
average,"  "average,"  or  "above  average. "  The  grade  is  obtained  by  summing  the 
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columns,  weighting  them,  and  computing  u  ratio  of  the  weighted  sum  to  the  total  number 
of  ctiecks  (omitting  the  "not  observed"  checks.)  The  grade  is  then  converted  into  a 
standard  score  according  to  a  frequently  revised  schedule.  In  turn,  the  standard  score 
is  combined  with  standard  scores  in  the  other  phases  of  training  to  obtain  the  final  grade 
for  the  stage  in  question  using  a  weighting  technique.  But  since  these  grades  are  com¬ 
piled  following  completion  of  the  study,  rather  than  at  a  time  when  they  could  influence 
evaluation  of  the  man,  it  is  not  likely  that  the  OLQ  rating  actually  has  much  formal 
effect  on  decisions  about  retaining  or  dropping  a  student. 

Practices  vary  widely  in  administering  the  OLQ,  in  the  number  of  persons  who  pre¬ 
pare  it,  who  fills  in  the  form,  and  in  what  factors  determine  the  rating.  Without 
going  into  detail,  we  may  indicate  the  variety  of  procedures  followed. 

a)  Number  of  raters.  This  varies  from  I  e.g,,(at  one  stage  of  basic  training  in  1952- 
53)  to  as  many  as  55-60  (e.g.,  in  certain  units  of  advanced  training  in  1952-53).  In 
pre-flight,  a  single  form  is  compiled  on  the  basis  of  reports  from  academic,  physical 
fitness,  and  survival  instructors,  and  from  military  authorities.  At  each  stage,  of 
course,  the  several  ratings  are  totalled  into  a  single  measure. 

b)  Who  does  tne  rating.  The  situation  for  pre-flight  has  already  been  mentioned; 
the  three  sources  of  judgment  are  assigned  different  weights,  with  the  military  looming 
largest.  At  the  other  stages  the  OLQ  rating  is  based  primarily  on  observations  during 
flight  training,  as  well  as  on  judgment  by  military  personnel,  and  ground  school.  In 
some  stages,  however,  only  the  flight  instructor  prepares  the  OLQ  form,  usually  the 
instructor  who  has  had  the  most  hops  with  the  student.  In  all  stages  past  pre-flight  the 
military  contribution  is  relatively  slight,  primarily  because  there  is  little  opportunity 
for  those  in  official  command  of  the  cadet  battalions  to  observe  the  student. 

c)  What  determines  the  ratings.  There  is  good  reason  to  syppose  that  most  of  those 
who  make  the  ratings  are  aware  of  the  fact  that  nontechnical  qualities  are  to  be  as¬ 
sessed  (i.e.,  in  contrast  to  the  academic  or  flight  proficiency  of  the  individual),  and 
that  they  conscientiously  strive  to  base  the  rating  on  these  qualities.  It  is  clear,  how¬ 
ever,  that  the  success  with  which  this  is  achieved  varies  from  place  to  place.  The 
criteria  by  which  nontechnical  qualities  are  judged  sometimes  have  the  character  of 
military  standards  (e.g.,  demerits  incurred  at  the  barracks),  at  other  times  of  academic 
grades  (especially  at  pre-flight),  and  at  still  other  times  of  personal  acquaintance 
(e.g.,  contact  at  the  flight  line).  It  is  not  knownwith  how  much  consistency  various 
possible  criteria  are  applied. 
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One  facet  of  the  general  problem  deserves  special  mention,  namely,  the  relation  of 
leadership  to  the  role  of  officer.  From  the  OLQ  form  itself  it  can  be  seen  that  the 
traditional  concept  of  the  officer  as  a  leader  of  men  is  accepted  by  including  an  item 
specifically  concerned  with  leadership.  We  cannot  here  enter  into  discussion  of  ;he 
vexing  problem  of  "leadership,"  its  meaning,  functions,  and  measurement.  With  par¬ 
ticular  regard  to  the  OLQ  rating,  however,  a  number  of  doubts  arise  concerning  the 
appropriateness  of  this  item.  For  one  thing,  if  is  found  that  a  very  high  proportion  of 
"not  observed"  judgments  occur  on  it,  confirming  the  common  sense  notion  that  most 
cadets  rarely  have  an  opportunity  to  function  in  situations  where  an  adequate  obser¬ 
vation  of  leadership  can  be  made.  There  is  the  further  point  that  leadership  qualities 
may  not  coincide  closely  enough  to  other  officer-like  qualities  to  regard  them  as 
equivalent;  rather,  one  might  expect  that  leadership  should  be  treated  as  an  entirely 
separate  factor.  Thus,  for  instance,  while  all  officers  can  reasonably  be  compared  on 
such  traits  as  military  courtesy  and  initiative,  it  may  be  pertinent  to  rate  on  leadership 
only  certain  officers  or,  more  likely,  all  officers  only  in  certain  situations  in  which  not 
everyone  is  called  updn  to  act.  In  general,  there  are  so  many  poorly  formulated  or 
conflicting  notions  about  the  relation  between  leadership  and  the  role  of  the  naval 
officer  that  careful  study  would  be  required  before  adequate  proposals  could  be  pre¬ 
sented.  It  does  not  seem  sufficient  to  assume  without  qualification  that  the  words 
"officer"  and  "leader"  are  essentially  synonymous.  The  newly  developing  leadership 
training  programs  in  naval  air  training  may  go  far  towards  clarifying  these  issues. 

Future  studies  from  this  laboratory  will  undoubtedly  prove  important  in  this  respect. 

We  shall  later  comment  on  some  possible  ways  in  which  the  assessment  of  officer-like 
qualities  can  be  improved.  As  will  appear  subsequently,  the  rating  now  possesses  con¬ 
siderable  validity  as  a  predictor  of  who  becomes  a  satisfactory  officer.  At  the  same 
time,  it  is  evident  from  the  foregoing  summary  that  no  clearly  defined  standards  are 
used  in  making  the  ratings.  Although  it  cannot  be  said  that  the  ratings  are  obtained 
haphazardly,  nevertheless,  there  is  a  large  element  of  chance  and  intuition  involved 
in  making  them.  There  is  some  reason  to  suppose  that  the  kind  of  judgment  required 
can  never  be  objectified,  at  least  in  the  situation  here  considered,  but  there  is  also 
good  reason  to  think  that  the  dependability  of  the  rating  can  be  much  improved. 

Finally,  it  may  be  worthwhile  to  mention  a  few  of  the  attitudes  which  seem  to  pre¬ 
vail  regarding  the  Ol.Q  rating  and  its  efficacy.  For  instance,  it  is  said  that  it  is  use¬ 
ful  in  identifying  the  outstanding  and  the  substandard  men,  and  our  own  results  certain¬ 
ly  bear  out  this  impression.  Of  the  greatest  significance  are  remarks  that  it  is  highly 
desirable,  or  essential,  to  assess  officer-like  qualities  in  some  way,  despite  consider¬ 
able  dissatisfaction  with  the  present  OLQ  rating  itself.  In  short,  there  can  be  little 
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doubt-  that  the  continuance  of  the  rating  somehow  expresses  a  widespread  conviction 
that  learning  how  to  fill  the  role  of  officer  is  a  very  important  component  of  the  train¬ 
ing  program.  How  satisfactory  measurement  is  to  be  achieved,  however,  encounters 
a  variety  of  proposals.  These  need  not  be  detailed  here,  but  it  might  prove  valuable 
to  conduct  a  separate  study  of  this  point.  In  general,  ideas  range  from  a  central  em¬ 
phasis  upon  strict  military  discipline  to  the  iniroduction  of  special  courses  in  how  to  be 
an  officer.  (  Incidentally,  these  same  proposals,  but  in  stronger  terms,  are  advanced 
as  a  means  to  improve  the  nontechnical  training  of  officer  students,  a  matter  which  will 
be  discussed  later.) 

PROCEDURE 

When  ways  were  sought  in  which  to  evaluate  the  OLQ,  there  were  ultimately  found 
to  be  several  sets  of  data  which  dould  properly  be  considered  pertinent.  The  first  of 
these,  of  course,  is  the  OLQ  itself  together  with  derivative  measures,  especially  the 
incidence  of  unsatisfactory  and  below  average  checks  on  OLQ  items.  The  second  con¬ 
sists  in  information  culled  from  the  training  record  which  appeared  to  express  the  non¬ 
technical  traits  in  question,  namely,  the  number  of  delinquency  reports,  the  number  of 
demerits,  and  holding  rank  as  cadet  officer.  Third,  there  are  peer  nominations  ob¬ 
tained  in  pre-flight.  Finally,  as  a  resuit  of  excellent  research  conducted  by  Lyon  and 
Berkshire  (8),  judgments  are  available  on  men  in  fleet  operations  in  terms  of  their 
satisfactoriness  as  officers.  These  last  data  were  used  as  criteria  against  which  to  val¬ 
idate  the  other  measures. 

The  central  problem  of  this  investigation  is  to  ascertain  the  validity  of  the  OLQ 
rating.  To  this  end  it  was  deemed  necessary  to  make  the  most  critical  analysis  possible, 
hence,  to  use  the  best  available  criterion  of  officerlikeness.  This  criterion  was  the 
judgment  of  this  general  characteristic  in  the  fleet,  as  further  described  below.  In 
accepting  this  criterion,  however,  it  was  necessary  to  make  use  of  a  sample  which  is 
not  large  by  ordinary  statistical  standards.  The  criterion  groups  included  i 1 2  men  (56 
in  each)  which  constitute  all  those  for  whom  the  fleet  criterion  was  available.  They 
are  a  reasonable  cross-section  of  the  men  who  were  performing  their  operational  train¬ 
ing  in  both  the  Pacific  and  Atlantic  Fleets,  at  the  time  the  data  were  collected.  The 
sample  includes  men  from  cadet  classes  entering  throughout  two  years,  1952  and  1953. 
No  class  is  represented  by  more  than  two  or  three  men. 


To  summarize  the  procedure,  it  might  be  described  as  the  most  unambiguously  de-  ‘ 

fined  test  case  of  the  OLQ  rating  which  could  be  devised  under  the  circumstances.  j 


It  should  not  be  assumed  tnat  no  furtner  research  is  necessary  or  possible  because, 
as  will  appear,  a  number  of  the  analyses  made  could  be  repeated  on  much  larger 
samples,  for  example,  relations  between  OLQ  scores  and  peer  nominations. 

Standard  statistical  procedures  were  used  to  solve  the  specific  problems  of  this 
study;  nence,  no  special  elaboration  of  them  is  required.  The  specific  problems  them¬ 
selves  will  be  evident  from  the  treatment  of  the  next  section. 

RESULTS 

Since  we  sought  answers  to  a  number  of  questions,  it  will  be  convenient  to  take 
them  up  one  at  a  time. 

I .  VALIDITY  OF  THE  OLQ  AND  OTHER  MEASURES 

The  major  problem  has  to  do  with  the  general  usefulness  of  the  OLQ.  That  is,  one 
wishes  to  know  whether  the  rating  has  an  identifiable  meaning  in  relation  to  its  in¬ 
tention.  To  answer  the  question  some  outside  criterion  of  officerlikeness  is  required. 
For  this  purpose,  as  previously  noted,  we  had  a  sampling  of  men  in  the  fleet,  for  each 
of  wnom  there  was  a  judgment  of  either  satisfactory  or  unsatisfactory  as  an  officer. 2 
Tne  judgment  was  solicited  from  the  s<juardron  commanding  officer,  usually  in  consul¬ 
tation  with  one  or  more  other  superior  officers.  It  is  this  measure  which  is  employed 
througnout  as  a  criterion  of  what  the  OLQ  rating  is  designea  to  predict  (hereinafter 
referred  to  as  "fleet  criterion").  The  relation  to  this  criterion  of  the  three  OLQ  stand¬ 
ard  scores,  at  pre-flight,  basic,  and  advanced  training,  together  with  other  measures 
similarly  evaluated,  is  shown  in  Table  I. 

It  is  apparent  that  a  rather  favorable  answer  to  the  question  is  provided,  taking  the 
picture  as  a  whole.  Throughout  training  there  is  a  strong  tendency  for  the  satisfactory 
men  to  differ  from  the  unsatisfactory  men.  It  is  evident  that  all  of  the  measures  are  in 
this  direction,  but  that  four  of  them  (OLQ,  Pre-Flight;  Leadership  peer  Nominations; 
Demerits;  Basic;  and  Demerits,  Advanced)  fail  to  reveal  significant  differences.  We  . 
have,  then,  preliminary  evidence  that  the  OL^  and  related  measures  have  validity  as 
predictors  of  the  fleet  criterion. 


*  Actually,  there  was  some  variation  in  procedure  between  the  Atlantic  and  Pacific 
Fleets;  in  both  cases,  our  Ss  fell  into  the  two  classes  mentioned.  Another  judgment 
was  also  obtained  regarding  calibre  as  a  flyer,  but  we  have  not  used  this  second 
criterion  in  the  present  study. 
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Before  continuing  with  questions  raised  by  the  results  in  Table  1,  we  should  say  a 
word  about  the  peer  nominations.  If  the  difference  between  tne  leadership  and  social 
living  choices  holds  up  in  further  investigations,  it  may  have  two  sorts  of  implications. 
In  the  first  place,  it  may  well  be  that  social  living  is  a  better  indicator  of  what  is 
measured  by  the  fleet  criterion  than  is  leadership.  In  this  case,  it  would  be  desirable 
to  employ  the  former  rather  than  the  latter  for  this  purpose.  The  second  implication 
bears  upon  the  distinction  drawn  earlier  among  the  several  classss  of  nontechnical 
requirements.  It  suggests  that  although  both  sorts  of  nominations  may  reflect  important 
aspects  of  satisfactory  officer  qualities,  they  are  far  from  identical  and  might  feasibly 
be  treated  as  separate  variables. 

One  other  genera!  point  requires  analysis,  namely,  the  possibility  that  ratings  of 
officer-like  qualities  are  influenced  by  a  man's  performance  in  the  technical  aspects 
of  training.  The  influence,  so  far  as  the  present  study  is  concerned,  may  have  been 
either  upon  the  OLQ  ratings  during  training,  or  upon  the  judgments  made  by  command¬ 
ing  officers  in  the  fleet. 

There  is  no  conclusive  way  to  check  upon  the  second  type  of  effect.  It  is  likely, 
however,  that  there  was  a  certain  amount  of  confusion  between  the  judgment  of 
"unsatisfactory  as  a  pilot"  and  "unsatisfactory  as  an  officer."  That  is,  a  substantia! 
proportion  of  the  men  in  our  sample  were  given  similar  judgments  on  both  factors. 

It  cannot  be  said,  however,  which  way  the  bias  operates,  whether  a  good  pilot  tends 
to  be  judged  satisfactory  as  an  officer,  or  whether  the  good  officer  tends  to  be  judged 
satisfactory  as  a  pilot. 

The  first  type  of  effect  would  show  itself  in  the  relation  between  ground  school 
and  flight  grades  during  training  and  the  fleet  criterion.  That  is,  if  OLQ  ratings  re¬ 
flect  technical  proficiency,  then  both  should  differentiate  between  men  judged  satis¬ 
factory  and  unsatisfactory  as  officers.  Table  II  presents  relevant  findings.  !t  is  clear 
that  a  tendency  toward  the  expected  influence  appears,  although  significant  only  in 
the  ground  school  grade  at  advanced  training.  As  pointed  out  above, here,  too,  we 
cannot  be  sure  whether  technical  proficiency  influences  grades  in  nontechnical  areas, 
or  vice  versa.  In  any  case,  the  one  is  not  entirely  independent  of  the  other.  Our 
data  suggest,  on  the  whole,  that  OLQ  ratings  express  something,  at  least,  over  and 
beyond  academic  and  flight  proficiency. 

In  general,  we  are  justified  in  concluding  that  there  are  significant  relations  among 
measures  of  nontechnical  qualities  obtained  early  in  training  and  subsequent  behavior, 
some  two  years  or  more  later,  under  operational  conditions. 


TABLE  II 


Ground  School  and  Flight'  Grades  in  Relation  to  the  Fleet  Criterion 


Grade  M 

Pre-Flight:  Ground  School 

Satisfactory  44.05 

Unsatisfactory  40.80 

Basic:  Ground  School 

Satisfactory  49.30 

Unsatisfactory  47.66 


Flight: 


Satisfactory  46.55 

Unsatisfactory  43.21 

Advanced:  Ground  School 

Satisfactory  44.82 

Unsatisfactory  40.45 


Flight: 

Satisfactory  46.36 

Unsatisfactory  47.05 


O  CR  P 

8.80 

10.42  1.79  .07 

8.48 

8.28  1.04  .20 

9.18 

9.70  1.87  .06 

7.67 

8.66  2.82  .01 


9.03 

5.66  .48 
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TABLE  III 


Tetrachoric  Interccrrelations  of  OLQ  Scores  with  Several  Measures  of 

Nontechnical  Qualities* 


Officerlike  Qualities  Rating 


Pre-Flight 

Basic 

Advanced 

Demerits,  PF 

.95 

.10 

.10 

Demerits,  Basic 

.02 

.13 

.10 

Demerits,  Adv. 

.22 

-.06 

-.25 

Delinquency  Rep: 

Pre-Flight 

.41 

.13 

-.05 

Basic 

-.10 

.15 

-.06 

Advanced 

.18 

-.08 

-.28 

Total  No.  U  &  BA 

.38 

.48 

.33 

*These  are  men  on  whom  peer  nominations  were  available,  representing  a 
shrinkage  from  the  total  fleet  criterion  groups. 
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PROPERTIES  OF  THE  OLQ 


In  order  to  interpret  more  clearly  the  usefulness  of  the  OLQ  rating,  it  becomes 
necessary  to  examine  it  more  closely,  for  several  reasons.  For  one  thing,  we  wish 
to  know  why  at  pre-flight  it  fails  to  distinguish  men  on  the  basis  of  the  fleet  criterion. 
A  second  question  pertains  to  its  relation  to  the  other  measureswhich  show  a  signif¬ 
icant  difference  in  terms  of  the  fleet  criterion  (e.g.,  its  relation  to  demerits).  Finally, 
as  a  rating  scale,  it  is  necessary  to  inquire  into  the  functions  of  the  various  items.  We 
shall  return  to  the  first  point  after  considering  the  other  two. 

Table  III  gives  intercorrelations  among  some  of  the  variables.  The  interpretation 
of  this  table  need  not  occupy  us  long.  It  is  apparent  that  the  number  of  demerits 
received  in  pre-flight  training  is  practically  snnonymous  with  the  OLQ  rating.  Other¬ 
wise,  there  is  little  relation  among  the  variables  save,  as  would  be  expected,  that 
the  number  of  below  average  and  unsatisfactory  checks  (the  latter  are  extremely  rare 
at  any  stage)  is  related  to  the  OLQ  rating.  This  raises  a  question  concerning  the 
possible  usefulness  of  such  factors  as  demerits.  Unfortunately,  this  is  complicated  by 
two  sorts  of  conditions.  On  the  one  hand,  it  is  clear  that  wide  variation  exists  in  the 
influence  of  these  records  upon  the  actual  preparation  of  the  OLQ.  Demerits  appear 
to  be  exceedingly  important  in  pre-flight  (or,  at  least,  both  demerits  and  the  OLQ 
are  determined  by  the  same  condition)  whereas,  if  anything,  the  opposite  influence 
occurs  at  the  advanced  stage  where  the  more  demerits  a  man  has  (within  limits)  the 
more  likely  he  is  to  be  rated  high  on  the  OLQ.  In  short,  despite  the  low  intercor¬ 
relations  it  does  not  seem  wise  to  treat  the  measures  as  if  they  were  independent.  The 
second  condition  is  that  a  change  in  the  modes  of  assessing  officer-like  qualities  might 
well  have  unexpected  consequences.  For  instance,  it  the  rating  of  a  cadet  as  a 
potential  officer  were  to  be  based  on,  let  us  say,  demerits,  perhaps  on  the  grounds 
that  these  have  more  concrete  character  than  the  OLQ,  then  the  relation  of  demerits 
to  the  fleet  criterion  might  change  markedly.  By  the  same  token,  placing  heavier 
reliance  upon  the  "U"  and  "BA  checks  (see  Table  IV)  would  have  to  be  assessed  follow¬ 
ing  a  period  when  the  change  has  had  a  chance  to  operate  in  the  ongoing  training 
context.  As  will  be  shown,  the  high  prediction  value  of  these  checks  is  partly  an 
unwitting  result  of  making  the  ratings;  that  is,  it  emerges  only  after  the  "mask"  of  the 
over-all  OLQ  rating  has  been  removed.  Most  men  later  judged  as  unsatisfactory  in 
the  fleet  obtain  above  average  marks  on  enough  items  to  bring  the  general  grade  close 
to  or  above  the  average.  It  is  only  by  examining  the  pattern  of  checks  that  the  dif¬ 
ference  between  those  later  adjudged  satisfactory  and  those  considered  unsatisfactory 
appears.  It  is  not  clear  just  what  would  happen  if  the  context  were  changed,  by 
elminating  the  average  and  above  average  columns. 
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TABLE  IV 


The  Difference  Between  "Satisfactory"  and  "Unsatisfactory" 
Men  in  Terms  of  OLQ  "Unsatisfactory"  and  "Below  Average" 
Checks  on  Any  Items  (N  =  56,  in  each  group.) 


Pre-Flight 

Basic 

Advanced 

No.  U 

No.  U 

%  U 

&  BA 

Sat.  Unsat. 

&  BA 

Sat. 

Unsat. 

&  BA* 

Sat. 

Unsat. 

5  or 

1  or 

1  or 

more 

1 

12 

more 

9  18 

more 

9 

23 

3-4 

4 

9 

0 

47  38 

0 

47 

33 

1-2 

9 

II 

M 

0 

0 

42 

24 

56  56 

56 

56 

56 

56 

X2  =  3.96 

X2  =  8.58 

X2  =  16.32 

n  =  1 

n  =  1 

n  =  3 

P.05 

P.OI 

P.OI 
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TABLE  V 


Distribution  of  Checks  on  Items  of  the  OLQ  Rating  for  Each  of  the  Three  Stages  of 
Training,  Showing  Contrasting  Patternsfor  the  Fleet  Criterion  Groups(in  percentages)* 


Fleet 


Stage 

Crit. 

NO 

U-BA 

A 

AA 

Item  1 

Pre-FI. 

Sat. 

0.0 

3.6 

50.0 

46.4 

Mii  itary 

Unsat. 

0.0 

8.9 

55.4 

35.7 

Courtesy 

Basic 

Sat. 

0.0 

.4 

59.6 

39.9 

Unsat. 

0.0 

l.l 

61.9 

37.0 

Adv. 

Sat. 

2.3 

.2 

63.5 

33.9 

Unsat. 

3.5 

1.9 

66.7 

27.9 

Item  2 

Pre-FI. 

Sat. 

0.0 

10.7 

39.3 

50.0 

Military 

Unsat. 

0.0 

23.2 

35.7 

41.1 

Benavior 

Basic 

Sat. 

.2 

l.l 

66.7 

32.0 

Unsat. 

0.0 

3.4 

65.5 

31.2 

Adv. 

Sat. 

8.4 

.5 

78.8 

12.3 

Unsat. 

5.8 

5.8 

74.7 

13.6 

Item  3 

Pre-FI . 

Sat. 

0.0 

3.6 

78.6 

17.9 

Military 

Unsat. 

0.0 

7.1 

75.0 

17.9 

Drill 

Basic 

Sat. 

61.2 

0.0 

36.3 

2.5 

Unsat. 

60.8 

.2 

37.0 

2.0 

Adv. 

Sat. 

90.4 

.1 

8.7 

.7 

Unsat. 

93.0 

.1 

6.0 

1.0 

Item  4 

Pre-FI . 

Sat. 

0.0 

0.0 

75.0 

25.0. 

initiative 

Unsat. 

0.0 

0.0 

76.8 

23.2 

Basic 

Sat. 

.2 

.4 

77.6 

21.8 

Unsat. 

.4 

2.2 

80.9 

16.4 

Adv. 

Sat. 

4.8 

2.5 

75.3 

17.4 

Unsat. 

6.4 

3.3 

73.4 

17.0 
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TABLE  V  Continued 

Fleet 


Stage 

Crit. 

NO 

U-BA 

A 

AA 

Item  5 

Pre-FI 

.  Sat. 

0.0 

0.0 

87.5 

12.5 

Soc  ial 

Unsat. 

1.8 

0.0 

66.1 

32.1 

Adeptness 

Basic 

Sat. 

12.9 

0.0 

73.7 

13.4 

Unsat. 

12.8 

.9 

77.3 

9.0 

Adv. 

Sat. 

58.9 

0.0 

33.3 

7.7 

Unsat. 

55.7 

2.0 

34.5 

7.7 

Item  6 

Pre-FI 

.  Sat. 

0.0 

0.0 

94.6 

5.4 

Stability 

Unsat. 

1.8 

0.0 

67.9 

30.4 

Basic 

Sat. 

.9 

.9 

86.0 

12.2 

Unsat. 

2.2 

2.2 

81.2 

14.3 

Adv. 

Sat. 

15.9 

1.0 

69.7 

13.3 

Unsat. 

16.3 

2.0 

73.0 

8.7 

Item  7 

Pre-FI 

.  Sat. 

0.0 

0.0 

62.5 

37.5 

Character 

Unsat. 

1.8 

0.0 

53.6 

44.6 

Basic 

Sat. 

0.0 

0.0 

61.9 

38.1 

Unsat. 

2.2 

.2 

61.9 

35.4 

Adv. 

Sat. 

48.2 

.5 

37.2 

14.0 

Unsat. 

76.2 

1.2 

12.2 

10.3 

Item  8 

Pre-FI 

.  Sat. 

0.0 

0.0 

80.4 

19.6 

Leadership 

Unsat. 

3.6 

1.8 

87.5 

7.1 

Basic 

Sat. 

39.2 

0.0 

56.9 

3.9 

Unsat. 

36. 5 

.4 

59.6 

3.4 

Adv. 

Sat. 

80.0 

.4 

18.2 

1.4 

Unsat. 

79.0 

« 

2.4 

17.4 

1.2 

*  Abbreviations  are 

to  be  read  as  follows 

:  "NO" 

-  Not  Observed; 

"U-BA" 

-  Unsatis- 

factory  and  Below  Average;  "A"  -  Average;  11 AA"  -  Above  Average. 
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TABLE  VI 


Distribution  of  Checks  on  Items  of  tne  OLQ  Rating  for  Each  of  the  Three  Stages  of 
Training,  Showing  Contrasting  Patterns  for  tne  High  and  Low  Thirds  of  tne  OLQ 

(in  percentages)* 


Stage 

OLQ 

NO 

U-BA 

A 

AA 

Item  1 

Pre-FI 

High 

0.0 

0.0 

19.4 

80.6 

Military 

Low 

0.0 

14.9 

74.5 

10.6 

Courtesy 

Basic 

High 

0.0 

0.0 

63.5 

36.5 

Low 

0.0 

1.3 

67.9 

30.8 

Adv. 

High 

6.2 

0.0 

38.0 

55.8 

Low 

1.9 

2.0 

74.1 

21.9 

Item  2 

Pre-FI. 

High 

0.0 

3.2 

35.5 

61.3 

Military 

Low 

0.0 

31.9 

34.0 

34.0 

Behavior 

Basic 

Hight 

0.0 

0.0 

67.9 

32.0 

Low 

.3 

5.4 

72.4 

21.8 

Adv. 

High 

6.8 

1.3 

64.9 

27.0 

Low 

6.8 

4.3 

82.2 

6.8 

Item  3 

Pre-FI. 

High 

0.0 

0.0 

51.6 

48.4 

Military 

Low 

0.0 

12.8 

85.1 

2.1 

Drill 

Basic 

High 

47.2 

0.0 

50.0 

2.8 

Low 

59.3 

.3 

39.4 

1.0 

Adv. 

High 

96.1 

.3 

3.2 

.3 

Low 

93.1 

0.0 

6*9 

0.0 

Item  4 

Pre-FI . 

High 

0.0 

0.0 

41.9 

58.1 

Initiative 

Low 

0.0 

0.0 

93.6 

6.4 

Basic 

High 

.6 

.3 

77.9 

21.3 

Low 

.3 

2.9 

85.9 

10.9 

Adv. 

High 

11.4 

.6 

52.9 

35.1 

Low 

3.8 

4.5 

8!. 9 

9.8 
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TABLE  VI  Continued 


Stage 

OLQ 

NO 

U-BA 

A 

AA 

Item  5 

Pre-FI . 

High 

0.0 

0.0 

71.0 

29.0 

Social 

Low 

0.0 

0.0 

74.5 

25.5 

Adeptness 

Basic 

High 

10.2 

.3 

75.1 

14.4 

Low 

11.2 

.6 

83.0 

5.1 

Adv. 

Hign 

38.0 

0.0 

35.7 

26.3 

Low 

61.0 

1.9 

33.9 

3.2 

Item  6 

Pre-FI. 

High 

0.0 

0.0 

83.9 

16.1 

Stability 

Low 

0.0 

0.0 

85.1 

14.9 

Basic 

High 

.3 

.3 

79.0 

20.2 

Low 

2.2 

2.2 

85.3 

10.3 

Adv. 

High 

16.2 

.3 

55.2 

28.2 

Low 

15.2 

1.8 

77.3 

5.7 

Item  7 

Pre-FI . 

Hign 

0.0 

0.0 

45.2 

54.8 

Cnaracter 

Low 

0.0 

0.0 

72.3 

27.7 

Basic 

High 

.6 

0.0 

61.9 

37.6 

Low 

1.3 

.3 

65.7 

32.7 

Adv. 

High 

29.2 

0.0 

31.2 

24.7 

Low 

51.7 

1.0 

42.9 

4.3 

Item  8 

Pre-F! . 

High 

0.0 

0.0 

58.0 

41.9 

Low 

2.1 

2.1 

93.6 

2.1 

Basic 

High 

27.6 

0.0 

67.1 

5.2 

Low 

40.1 

.6 

57.7 

1.6 

Adv. 

High 

76.6 

0,0 

20.8 

2.6 

Low 

75.8 

2.4 

20.6 

J.l 

*See  Table  V  for  abbreviations 
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The  manner  in  which  the  individual  items  are  checked  is  shown  :n  Tables  V  and  VI. 
In  the  former  the  two  fleet  criterion  groups  are  compared,  and  in  the  latter  the  top 
and  bottom  thirds  of  OLQ  standard  scores  at  each  stage  of  training.  No  detailed 
analysis  is  required  to  make  clear  the  several  points  which  emerge.  In  the  first  place, 
it  is  noteworthy  that,  with  few  exceptions,  there  is  general  agreement  between  the 
two  Tables.  The  chief  difference,  as  might  be  expected,  is  that  the  high  and  low 
thirds  are  throughout  much  more  sharply  differentiated  than  are  the  two  fleet  criterion 
groups.  Thas  is  partly  accounted  for,  of  course,  by  the  fact  that  the  middle  group  is 
omitted  in  the  first  case  but  divided  among  the  two  groups  in  the  second  case.  It  is 
also,  however,  and  this  is  a  crucial  point,  accounted  for  by  the  fact  that  above  aver¬ 
age  and/or  average  checks  are  assigned  more  feequently  on  certain  items  to  the  fleet 
unsatisfactory  men  than  to  the  fleet  satisfactory  men.  The  result,  of  course,  is  to 
make  the  average  difference  smaller.  Here  lies  the  reason  why  the  pre-flight  OLQ 
rating  fails  to  distinguish  the  criterion  groups.  Were  only  the  first  three  and  the  last 
items  to  be  included  in  the  socre,  the  criterion  groups  would  be  clearly  different. 

Note  that  ail  of  the  U-BA  checks  occur  on  these  items  lending  credence  to  the  sup¬ 
position  that  an  effort  is  somehow  made  to  compensate  bad  checks  on  one  item  with 
above  average  checks  on  another. 

In  the  second  place  it  is  quite  clear  that  certain  items  contribute  little  or  nothing 
to  the  total  OLQ  rating.  Thus,  the  proportion  of  "not  observed"  judgment  on  items 
3,  5,  and  8  at  either  basic  or  advanced  stage,  or  both,  is  very  high.  Furthermore, 
;ome  items  show  such  small  differences  in  either  Table  as  to  render  doubtful  their 
function  in  measuring  the  trait  in  question;  items  4  and  5  are  especially  to  be  remarked 
in  this  respect.  The  concurrence  of  these  two  phenomena  suggests  that  some  better 
combination  of  items,  or  some  other  rating  device,  would  prove  such  more  efficacious 
than  the  currently  used  OLQ  form. 

3.  THE  PATTERN  OF  OFFICER-LIKE  QUALITIES 

In  view  of  the  foregoing  analyses,  it  is  evident  that  throughout  the  training  program 
those  more  likely  to  become  satisfactory  officers  differ,  on  the  average,  from  those 
who  ultimately  appear  in  the  fleet  as  unsatisfactory  in  this  respect.  It  is  also  clear 
that  considerable  improvement  could  be  achieved  in  the  measurement  of  the  nontech¬ 
nical  qualities  involved.  As  a  means  to  bring  out  the  contrasting  character  of  crite¬ 
rion  groups  at  earlier  points  in  their  careers,  an  effort  was  made  to  compose  patterns  of 
the  variables  which  have  been  analysed.  This  approach  is  merely  suggestive,  because 
the  measures  have  been  selected  after  the  analysis  has  been  made. 

We  sought,  on  tne  one  hand,  to  combine  measures  wnich  were  both  significant  and 
nonoverlapping  and,  on  the  other  hand,  to  retain  the  predictive  function  of  the  meas¬ 
ure  (that  is,  to  look  from  the  earliest,  or  pre-flight,  stage  to  later  ones  since  it  would 
not  be  of  great  practical  assistance  to  base  a  prediction  on  measures  secured  during 
advanced  training.)  It  is  plain  that  pre-flignt  measures  offer  the  greatest  problem 
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because  of  the  nonsignificance  of  the  difference  in  OLQ  ratings  at  that  stage. 

To  form  the  patterns  each  variable  was  dichotomized  ai  u  point  of  division  which 
differentiated  the  criterion  groups  without  too  great  a  disproportion  in  tb e  subsamples. 
The  divisions  were  made  as  follows: 

Cadet  Officer:  Yes  versus  No 

OLQ,  Basic:  Above  or  below  the  mean  of  the  Fleet  unsatisfactory  sample. 

No.  of  U-BA  checks.  Advanced:  0  to  2  per  cent  versus  3  per  cent  or  more. 

(A  percentage  figure  was  used  for  his  measure  because  of  the  extremely  wide 
variation  in  the  number  of  judgments  among  the  men.  Thus,  although  all  men 
received  the  same  number  at  pre-flight  and  basic;  at  advanced  the  range  was 
from  about  32  (4  OLQ  ratings  with  8  items  each)  to  as  many  as  448  (56  OLQ 
ratings.) 

No.  of  U-BA  checks,  Pre -Flight:  0  versus  I  or  more. 

No.  of  U-BA  checks,  Basic:  0  versus  I  or  more. 

Peer  Nominations  (Leaderhhip),  Pre-Flight:  Above  or  below  the  mean  of  the 
Fleet  unsatisfactory  sample. 

A  score  above  the  cutting  point  on  each  variable  was  assigned  a  "plus,"  a  score 
below  that  point  a  "minus."  The  number  of  plusses  received  by  each  man  was  counted. 
The  incidence  of  the  number  of  plusses  was  then  determined.  In  Table  VII  is  shown 
for  each  criterion  group  the  number  cf  men  who  received  various  numbers  of  plusses  for 
three  combinations  of  variables.  It  can  be  seen  that  all  three  patterns  differentiate 
between  criterion  groups. 

That  is  is  a  consistent  rather  than  merely  an  average  difference  is  shown  graph¬ 
ically  in  Figures  I  and  2.  These  graphs  are  designed  to  show  how  permormance  at  early 
stages  of  training  wos  related  in  this  sample  to  the  fleet  criterion.  Men  who  are  sub¬ 
standard  in  bo*h  pre-flight  and  basic  are  more  infrequently  rated  unsatisfactory  a  year 
or  so  later  in  the  fleet.  If  two  independent  measures  are  employed  at  the  pre-flight 
stage  (see  Figure  2),  then  this  earliest  stage  alone  permits  one  to  distinguish  sharply 
between  the  two  criterion  groups. 


TABLE  VII 


Patterns  of  Difference  Between  Fleet  Criterion  Groups 


I 

i  * 


2. 


3. 


Using  as  measures  (a)  cadet  officer,  Pre-Flight,  (b)  OLQ  rating  in  Basic,  and 
(c)  "U-BA"checks  in  Advanced. 


No.  of 

Fleet  Criterion 

No.  Expected 

Plusses 

Sat. 

Unsat. 

%  Sat. 

in  Unsat. 

3 

12 

3 

26 

1114 

2 

22 

15 

49 

21.6 

1 

10 

35 

24 

10.6 

0 

1 

11 

45 

44 

S2  =  29.64 

n  =  2 

P  .01 


Using  as 

measures  (a)  U-BA, 

Pre-Flight, 

(b)  U~BA,  Basic,  (c)  U-BA,  Advanced 

No,  of 

Fleet  Criterion 

No.  Expected 

Plusses 

Sat. 

Unsat. 

%  Sat. 

in  Unsat. 

3 

33 

14 

73 

32.1 

2 

7 

14 

16 

7.0 

1 

5 

13 

11 

4.8 

0 

0 

3 

45 

44 

X2  =  43.34 
n  =2 

P  .01 

Using  as  measures  (a)  peer  nominations  (leadership),  Pre-Flight,  (b)  "U-BA:1, 


Pre-Flight, 

(c)  OLQ,  Basic, 

(d)  "U-BA", 

Advanced. 

No.  of 

Fleet  Cri 

terion 

No.  Expecl 

Plusses 

Sat. 

Unsat. 

%  Sat. 

in  Unsat. 

4 

11 

3 

24 

10.6 

3 

18 

15 

40 

17.6 

2 

11 

7 

24 

10.6 

1 

5 

12 

11 

4.8 

0 

0 

7 

45 

44 

X2  =  47.85 

n  =  3 

P  .01 
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NUMBER  OF  MEN 


traits: 


MINUS  AT  BOTH  PLUS  AT  EITHER  PLUS  AT  BOTH 
PRE-FL.  ft  BASIC  PRE-FL.  OR  BASIC  PRE~FL.  &  BASIC 


FIGURE  I 

DIFFERENCE  BETWEEN  FLEET  CRITERION  GROUPS  ON 
THREE  MEASURES  OF  NON-TECH  N I  CAL  TRAITS 


■i 

i 


mm/m  mmm  . mrnrn  _ 

MINUS  AT  PLUS  AT  PRE-FL.  PLUS  AT  BOTH 
PRE-FLIGHT  BUT  MINUS  AT  PRE-FL.  a  BASIC 


BASIC 

‘’'A^plus'  on  either  of  these  was  accepted. 

FIGURE  2 

DIFFERENCE  BETWEEN  FLEET  CRITERION  CROUPS 
ON  FOUR  MEASURES  OF  NON-TECHNICAL  TRAITS 


DISCUSSION 


We  have  said  that  the  study  reported  here  represents  a  test  case.  It  has  shown 
that  the  OLQ  rating  as  it  was  made  during  the  years  1952  and  I'  ‘2  possessed  sufficient 
validity  to  indicate  that  it  was  capable  of  predicting,  on  the  average,  which  men  in 
training  were  likely  to  prove  satisfactory  or  unsatisfactory  as  officers  in  the  fleet.  The 
fact  that  nontechnical  qualities  can  be  assessed  satisfactorily  was  further  confirmed  by 
significant  differences  among  criterion  groups  in  several  other  measures  deemed  to  re¬ 
flect  these  qualities.  These  latter  measures  included  peer  nominations  in  pre-flight, 
having  held  a  cadet  officership  in  pre-flight,  and  the  record  of  delinquency  reports 
and  demerits.  The  analysis  of  all  of  these  measures  leads  to  substantially  the  same 
conclusion:  Men  later  judged  satisfactory  as  officers  appear  to  differ  in  general  ways 
from  the  beginning  of  their  training  from  those  later  considered  unsatisfactory  as  of¬ 
ficers.  Whether  or  not  the  difference  could  be  stated  as  a  pattern  of  adequacy  as  a 
person,  or  as  adaptation  to  the  naval  aviation  milieu,  or  in  some  other  fashion,  re¬ 
mains  to  be  decided  by  future  investigation.  For  the  present,  we  must  confine  our¬ 
selves  to  the  specific  findings  in  this  study. 

It  should  be  remembered  that  our  conclusions  are  based  upon  the  situation  as  it 
existed  when  the  men  in  our  sample  were  undergoing  training*  From  this  point  of  view 
the  really  st. iking  feature  of  our  results  is  that  the  OLQ  rating  should  have  different¬ 
iated  as  well  as  it  did  among  the  criterion  groups.  Even  in  the  essentially  chaotic  con¬ 
ditions  under  which  it  was  administered  and  with  many  serious  limitations  under  which 
the  assessment  of  the  rates  took  place,  nevertheless  the  OLQ  could  have  served  as  a 
fair  predictor  of  the  nontechnical  qualities  which  it  was  designed  to  measure.  Rather 
than  reiterate  these  points  then,  it  may  be  more  fruitful  to  suggest  some  concrete  ways 
in  which  the  method  of  assessing  nontechnical  aspects  of  training  can  be  improved. 

Let  us  turn  first  to  the  actual  rating  itself.  From  analysis  of  the  separate  items 
it  appears  that  the  eight  included  in  the  present  form  are  of  decidedly  unequal  effi¬ 
ciency.  Several  are  seldom  observed,  others  do  not  differentiate  between  good  and 
poor  men,  and  some  display  a  heavy  preponderance  of  average  and  above  average 
checks.  At  the  very  least,  therefore,  it  might  be  desirable  to  simplify  the  rating  by 
reducing  the  number  of  items,  if  no  further  attempt  is  to  be  made  to  locate  suitable 
ones. 

One  might  abandon  entirely  the  notion  of  rating  separately  each  of  a  congeries 
of  somewhat  vaguely  defined  traits  and,  instead,  seek  to  obtain  merely  an  over-all 
or  "halo"  rating.  This  proposal,  however,  must  be  rejected  for  several  reasons.  In 
the  first  place,  it  would  be  difficult  to  formulate  a  general  quality  which  would  be 


The  lack  of  standard  policy  with  respect  to  the  OLQ  rating  makes  it  impossible  to  say 
whether  validity  has  remained  stable. 
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meaningful,  ever  if  a  single  judgment  could  be  shown  to  be  reliable.  In  the  second 
place  there  are  purely  practical  objections.  Raters  prefer  to  have  a  series  of  separate 
items  in  order  to  justify  their  impressions.  Furthermore,  the  ratee  himself  needs  to 
have  specific  explanations  for  the  rating  when  it  is  presented  to  him.  Finally,  there 
are  quite  legitimate  variations  among  raters  in  the  factors  which  they  believe  them¬ 
selves  qualified  to  assess. 

A  more  feasible  approach  would  be  to  recognize  the  probability  that  officer-like 
qualities  include  the  three  or  four  different  factors  mentioned  earlier,  namely,  mili¬ 
tary  behavior,  leadership,  social  acceptability,  and  devotion  to  duty*with?n  the 
military  setting,  and  to  develop  items  bearing  upon  each  of  them.  Were  there  to  be 
three  or  four  items  in  each  area,  the  task  of  rating  could  be  made  sufficiently  concrete 
and  vaired  to  meet  the  points  mentioned  above.  What  is  more,  a  device  of  this  kind 
could  be  used  in  three  different  ways,  thus  increasing  the  flexibility  of  administering 
the  OLQ. 

1)  The  total  OLQ  score  could  simply  be  a  sum  of  the  judgments,  just  as  is  now  the 
case.  The  additional  advantage  of  the  proposed  system  would  be  a  greater  clarifica¬ 
tion  of  what  is  to  be  assessed,  together  with  an  increase  in  reliability  (for  example, 
there  would  be  several  leadership  items,  instead  of  only  one.) 

2)  Sub-scores  could  be  determined  in  each  area,  separately.  This  would  make  it 
easier  for  the  rater  to  make  judgments  only  in  those  areas  which  he  felt  competent  to 
deal  with;  in  fact,  raters  could  be  instructed  to  fill  out  only  the  part  of  the  form  which 
pertained  to  a  particular  stage  (e.g.,  military  behavior  at  pre-flight.)  The  separate 
parts  could  be  reported  as  separate  scores,  or  combined,  as  in  the  first  technique,  into 
an  over-all  score. 


3)  A  parallel  form  could  be  used  for  officers  or  aviation  officer  candidates,  per¬ 
haps  employing  a  slightly  different  phraseology  and  title,  t<o  avoid  imputing  the  same 
status  to  them  as  to  aviation  caaets,  and  also  omitting  items  not  applicable  to  them. 
The  problem  of  assessing  officer  candidates  is  generally  recognized  to  be  of  sufficient 
gravity  to  warrant  the  trying  out  of  a  procedure  of  this  kind.  These  men  are  supposed 
to  be  evaluated  not  alone  in  terms  of  then  potentiality  as  aviators  but  also  explicitly 
in  terms  of  their  promise  as  officers.  It  is  hard  to  see  how  a  definite  rating,  consci¬ 
entiously* made,  can  be  avoided.  !t  is  likely,  however,  that  administrative 

r 

*Anotner  way  to  phrase  this  quality  might  be  "identification  with  the  Navy." 


**Tne  word  "conscientiously"  is  employed  because  there  is  evidence  that  men  who  are 
expected  to  be  commissioned  at  the  end  of  pre-flignt  are  rarely  given  unfavorable 
ratings  on  the  OLQ  form.  However,  if  it  were  known  that  even  a  small  proportion 
of  men  would  be  eliminated  as  a  result  of  poor  ratings  on  the  OLQ  rating,  this  con¬ 
dition  might  change. 
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requirements  might  make  advisable  a  somewhat  different  form  for  officers.  In  this 
connection,  it  is  worth  mentioning  that  obtaining  OLQ  ratings  on  officers  in  training 
would  have  value  in  addition  to  the  customary  fitness  report.  The  latter  represents 
a  serious  commitment  since  it  is  a  permanent  record,  but  the  OLQ  rating  could  be  a 
temporary,  less  serious,  judgment.  It  could  also  be  useful  information  to  facilitate 
the  preparation  of  the  fitness  report. 

An  important  issue  which  must  be  considered  in  seeking  to  improve  the  measure¬ 
ment  of  officer-like  qualities  arises  because  of  the  relation  between  grades  in  tech¬ 
nical  and  nontechnical  areas.  That  is,  a  man  with  a  high  grade  in  one  is  likely  to 
some  extent  to  obtain  a  high  grade  in  the  other  (or  vice  versa).  It  is  not  known  in 
which  direction  the  effect  operates.  For  example,  a  cadet  who  possesses  outstanding 
qualities  as  a  potential  officer  may  for  this  reason  (but  perhaps  without  insight  on  the 
part  of  the  rater)  be  highly  rated  as  a  pilot  or,  it  may  work  the  other  way  ar  that  a 
cadet  who  is  an  excellent  pilot  may  be  more  highly  regarded  as  an  officer  than  one  who 
is  a  poor  pilot.  It  is  entirely  possible  that  both  kinds  of  effects  occur. 

The  further  possibility  cannot  be  discounted  that  the  relation  is  not  a  bias,  or 
contamination,  in  the  grading  system,  but  a  function  of  an  actual  correlation.  Men 
who  are  outstanding  or  substandard  in  one  respect  may  really  be  similar  in  the  other. 

It  could  be  a  function,  for  instance,  of  ability  or  movtivation.  So  far  as  assessment 
goes  persons  with  respect  to  whom  all  measures  are  in  agreement  pose  no  problem. 

It  is  rather  the  remainder  we  need  to  worry  about.  Among  them,  our  data  show,  are 
cadets  who  are  unsatisfactory  in  nontechnical  qualities. 

Ideally,  we  need  independent  measures  of  technical  and  nontechnical  qualities. 
From  a  practical  point  of  view  this  can  probably  not  be  achieved  in  any  economical 
fashion.  For  example,  it  would  appear  that  it  will  always  be  necessary  for  *he  same 
superior  officer  (a  flight  instructor,  let  us  say)  to  rate  a  man  Sn  both  technical  and 
nontechnical  qualifications.  For  this  reason  the  best  approch  would  be  to  strive  for 
the  greatest  possible  clarity  regarding  the  qualities  to  be  judged  and  instructions  which 
make  maximally  objective  the  task  of  performing  the  judgments.  The  suggestions  which 
this  research  has  brought  to  light  may  contribute  to  these  ends. 

If  we  turn  now  to  the  process  of  rating  itself,  a  number  of  points  should  be  made. 
For  one  thing,  it  is  probable  that  the  instructions  associated  with  the  rating  could  be 
formulated  in  a  manner  which  would  increase  the  validity  and  consistency  of  the 
judgment.  For  example,  the  tendency  to  "counterbalance"  unfavorable  checks  with 
favorable  ones  might  be  reduced.  Raters  could  be  urged  to  assign  checks  only  when 
they  are  confident  of  having  pertinent  information  on  the  ratee.  Otherwise,  checks 
should  be  assigned  to  the  not  observed  category.  The  policy  now  especially  evident 
in  advanced  training,  and  to  a  lesser  degree  in  basic  training,  of  obtaining  ratings 
from  a  large  number  of  raters  could  be  reinforced. 


The  question  of  who  is  best  qualified  to  make  the  ratings  deserves  further  study. 

It  appears  that  the  military  authorities  in  pre-flight  and  the  flight  instructors  at  other 
stages  are  probably  best  qualified,  and,  cf  course,  steps  have  recently  been  taken  in 
agreement  with  this  conclusion.  There  is  little  advantage  in  "cluttering  up"  the  rating 
with  judgments  from  sources  other  than  those  which  are  most  appropriate. 

Another  point  is  that  the  strong  validity  of  U-B/  checks  suggests  that  these  should 
be  assigned  a  higher  weight  in  computing  the  OLQ  grade.  Incidentally,  the  extreme 
rarity  of  U  checks  at  all  three  stages  of  training  indicates  that  they  might  be  eliminated 
unless  evidence  can  be  adduced  that  they  serve  to  anchor  the  judgments. 

Finally,  our  pattern  analysis  leads  to  another  possible  way  to  improve  the  measure¬ 
ment  of  officer-like  qualities.  One  might  develop  a  more  complex  score  than  that 
represented  by  the  OLQ  rating  itself.  Thus,  additional  values  might  be  given  for 
having  served  as  a  cadet  officer,* ** for  extremely  high  or  low  scores  on  the  peer  nomina¬ 
tion  test,  and  for  esipecially  frequent  delinquency  reports  or  demerits. 

Along  these  lines  valuable  additional. light  on  a  cadet's  nontechnical  traits  could 
be  obtained  by  an  extension  of  the  peer  nomination  technique  in  two  directions.  In 
the  first  place,  little  or  nothing  has  been  done  yet  with  peer  nominations  at  stages 
beyond  pre-flight.  Work  is  needed  especially  at  basic  where  the  measures  employed 
In  the  present  study  have  been  least  useful .  The  argument  that  cadets  do  not  know 
each  other  as  well  at  Whiting  Field,  for  instance,  as  at  pre-flight  is  weak.  Although 
it  is  true  that  there  is  much  more  diffuse  organization  and  mutual  acquaintance  after 
pre-flight,  nevertheless,  most  cadets  undoubtedly  size  up  their  associates  wherever 
they  are. 


The  second  extension  applies  to  superior  officers  sho  could  rank  the  cadets  with 
whom  they  are  acquainted?*  For  example,  a  flight  insturctor,  although  able  to  judge 
only  his  own  students, *is  in  a  good  position  to  compare  them  with  each  other  and  in 
this  manner  to  aid  in  identifying  those  who  possess  more  and  less  degree  of  officerlike- 
ness.  Indeed,  the  personal  contact  of  the  flight  line  is  perhaps  the  most  favorable 
condition  of  all  those  in  military  training  to  obtain  the  requisite  knowledge.  The 
necessity  to  rank  students  would  have  several  advantages  over  the  mere  rating  of  each 
one  separately.  It  would  force  attention  upon  the  judgment  and  call  upon  specific 


*  There  is  a  "chicken-or-fhe-egg"  problem,  here,  which  soould  be  investigated.  It 
is  not  know  whether  men  are  chosen  as  cadet  officers  because  they  have  a  higher  de¬ 
gree  of  "officerlikeness"  to  begin  with,  or  whether  they  ultimately  prove  to  be  satis¬ 
factory  as  officers  in  the  fleet  because  of  experience  acquired  as  a  cadet  officer  early 
in  training. 

**Steps  in  this  direction  appear  to  have  been  taken  already,  at  least  in  requesting  that 
academic  instructors  at  pre-flight  nominate  the  men  who  are  best  and  worst  in  officer¬ 
like  qualities. 
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experiences  with  the  cadet,  thus  fending  to  reduce  reliance  upon  the  average  or 
vague  rating.  If  would  also  provide  a  point  of  reference  for  the  judgment  since  each 
man  would,  of  necessity,  be  compared  with  at  least  one  other  man.  Such  rankings 
would  not  necessarily  supplant  the  OLQ  rating,  but  might  at  least  assist  in  making  it 
and  might  even  be  employed  as  a  weight,  e.g.,  by  increasing  or  decreasing  the 
over-all  rating  by  adding  and  subtracting  points  for  high  and  low  ranks. 

In  sum,  then,  various  means  to  improve  the  OLQ  rating  can  be  discerned.  It 
would  appear  fhet  nontechnical  qualities  are  of  sufficient  importance  in  the  scheme 
of  naval  life  that  they  deserve  serious  attention  and  implementation.  The  tenor  of  this 
report  is  that  men  who  ultimately  are  regarded  as  unsatisfactory  officers  in  the  fleet 
can  be  identified  with  considerable  accuracy  as  early  as  the  pre-flight  stage  of  train¬ 
ing.  While  the  nontechnical  qualities  involved  should  not  be  allowed  to  overshadow 
the  other  primary  requirements  of  training,  nevertheless,  they  appear  to  be  of  sufficient 
Importance  to  merit  special  attention.  At  least  a  small  proportion  of  trainees  who  are 
very  likely  to  prove  undesirable  from  this  point  of  view  might  well  be  eliminated 
without  much,  if  any,  loss  in  the  technical  qualities  of  the  Naval  Air  Force. 

Summing  up  briefly,  this  investigation  indicates  that  officerlikeness  can  be 
assessed  in  valid  ways  quite  early  in  training.  Possible  ways  to  improve  this  assessment 
have  been  discussed,  both  through  modidications  of  the  OLQ  rating  itself  and  through 
the  utilization  of  other  measures  such  as  peer  nominations,  delinquency  reports,  and 
performance  as  a  cadet  officer. 


RECOMMENDATIONS 

1.  A  serious  effort  should  be  made  to  improve  the  assessment  of  officer-like  qualities, 
both  in  the  direction  of  increasing  the  validity  of  measures  and  ofextending  the  in¬ 
formation  to  be  derived  from  these  measures. 

2.  Attention  should  be  given  to  the  items  upon  which  assessment  is  to  be  based. 

3.  a.  There  should  be  a  clarification  of  the  qualities  understood  by  the  term 

officerlike.  Tentatively,  we  propose  that  a  sisdinction  be  drawn  among 
military  behavior,  leadership,  acceptability  as  a  person  (i.e,,  a  mili¬ 
tary  associate),  and  devotion  to  duty  (or  identification  with  the  Navy). 

b.  In  developing  more  adequate  measures  of  officerlikeness,  these  categories 
should  be  treated  separately,  at  least  until  their  interrelations  and  contri¬ 
bution  to  the  over-all  officer  role  are  better  understood. 
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c.  Several  specific  Items  should  be  devised  in  each  of  the  four  general  areas. 

d.  Items  to  be  included,  whether  those  now  in  use  or  those  to  be  employed  in 
the  future,  should  meet  the  practical  requirements  of  rating  scales.  Those 
which  are  seldom  observed,  which  do  not  differentiate  amoung  good  and 
poor  men,  or  in  which  there  is  a  great  preponderance  of  average  or  above 
average  checks,  should  be  modified,  or  eliminated. 

e.  Consideration  should  be  given  to  the  assignment  of  higher  weight  to  below 
average  checks. 

3.  Steps  should  be  taken  to  improve  the  administration  of  the  OLQ  rating. 

a.  Instructions  should  be  strengthened  to  reduce  the  incidence  of  counter¬ 
balancing  substandard  checks  by  above  average  checks. 

b.  Wherever  possible,  more  than  one  rater  should  make  judgments  on  the 
same  student. 

c.  Decisions  should  be  reached  regarding  the  persons  best  qualified  to  prepare 
the  rating,  taking  into  account  the  fact  that  such  persons  may  differ  at 
various  stages  of  training. 

d.  Raters  should  be  urged  to  utilize  the  not  observed  category  when  they  lack 
confidence  in  their  ability  to  make  the  rating. 

e.  Every  effort  should  be  exerted  to  establish  uniform  procedures,  taking  into 
account  the  possibility  that  these  procedures  may  differ  at  aarious  stages 
of  training . 

4.  The  whole  approach  to  the  assissment  of  officerlikeness  should  be  broadened  to 

include  measures  in  addition  to  the  OLQ  rating  itself.  Special  consideration 

in  this  regard  liould  be  given  to  peer  nom’inations,  rankings  by  instructors, 

and  performance  as  cadet  officer. 

5.  Flexibility  should  be  introduced  in  a  manner  which  permits  the  seaprate  determ- 

intation  of  scores  (or  sub-scores)  for  different  aspects  of  officerlikeness. 
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6.  Forms  should  be  developed  for  the  assessment  of  officers  in  training  and  aviation 
officer  candidates,  both  to  facilitate  their  evaluation  as  future  officer  candidates, 
both  to  provide  useful  information  preliminary  to  preparation  of  fitness  reports. 

7.  Specific  administrative  action  should  be  taken  to  implement  the  assessment  of 
officerlikeness.  This  might  take  the  form  of  hearings  before  boards  or  an  actual 
elimination  from  training  for  unsatisfactory  ratings. 
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Appendix  A 


Sample  of  The  Officer-Like  Qualities  Rating  Form 


Omitted  is  the  lower  section  which  contains  computation  of  the  numberical  grade, 
date,  name,  unit,  and  other  information  of  an  administrative  character.  It  shall  be 
tne  responsibility  of  the  O-in-C  of  each  Training  Unit  to  insure  that  an  Officer-Like 
Qualities  report  is  placed  in  the-  Aviation  Training  Jacket  of  each  student  at  the  com¬ 
pletion  of  each  unit  of  training.  These  reports  shall  be  completed  in  accordance  with 
currect  CMATra  Air  Training  Instructions.  When  rating  student,  carefully  consider  and 
keep  in  mind  the  following  definitions: 

Unsatisfactory — Inefficient,  below  minimum  standards;  Below  Average- 'Satis¬ 
factory,  passably  efficient;  Average — Efficient,  well  qualified; Above  Average — 
Superior,  exceptionally  efficient;;  Not  observed — Use  irf  all  cases  where  tnere  nas  been 
insufficient  opportunity  to  observe. 

The  appropriate  column  will  be  checked  (  )  by  the  rating  officer. 


Not  Unsatis-  Below  Above 

Observed  factory  Average  Average  Average 


MILITARY  COURTESY 

Expresses  respect  and  politeness  in 
accordance  with  established 

Military  custom. 

MILIARY  BEHAVIOR 

Conforms  to  military  rules  and 
reguaations. 

MILITARY  DRILL 

Progress  and  facility  in  formation, 
drills,  and  inspections. 

INITIATIVE 

On  the  |ob,  interest,  does  his  best, 
desires  to  learn. 

SOCIAL  ADEPTNESS 

Cooperative,  considerate,  gets  along 
well  with  others,  respects,  thinks  of 
others. 

STABILITY 

Se!f  controlled,  even  tempered,  con¬ 
sistent  in  his  actions,  confident, at  ease. 

CHARACTER 

Dependable,  hones*,  reliable,  loyal. 

LEADERSHIP 

Demands  and  maintains  confidence  of 
others  so  that  they  will  act  uniformly 
upon  an  idea  or  mode  of  action. 

TOTAL  MARKS  THIS  SHEET 
CUMULATIVE  TOTALS 

« 
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